An Overview of Data-Driven Part-of-Speech Tagging

Over the last twenty years or so, the approaches to partof-speech tagging based on machine learning techniques have been developed or ported to provide high-accuracy morpho-lexical annotation for an increasing number of languages. Given the large number of morpho-lexical descriptors for a morphologically complex language, one has to consider ways to avoid the data sparseness threat in standard statistical tagging, yet to ensure that the full lexicon information is available for each wordform in the output. The paper overviews some of the major approaches to part-of-speech tagging and touches upon the tagset design, which is of crucial importance for the accuracy of the process. Key-words: ambiguity class, data sparseness, lexical ambiguity, machine learning, multilinguality, part-of-speech tagging, tagset design.